Papers with deep fusion
SWAFN: Sentimental Words Aware Fusion Network for Multimodal Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies focus on learning the joint representation of multiple modalities, ignoring useful knowledge contained in language modal. |
| Approach: | They propose to incorporate sentimental words knowledge into the fusion network to guide the learning of joint representation of multimodal features. |
| Outcome: | The proposed method improves the fusion representation of multimodal features on a YouTube and video dataset. |
Original Semantics-Oriented Attention and Deep Fusion Network for Sentence Matching (D19-1)
Copied to clipboard
| Challenge: | Sentence matching is a key issue in natural language inference and paraphrase identification. |
| Approach: | They propose a semantics-oriented attention and deep fusion network (OSOA-DFN) that is oriented to the original semantic representation of another sentence and propagates attention information at each matching layer. |
| Outcome: | The proposed model can model sentence matching more precisely on three sentence matching benchmark datasets. |
CLFFRD: Curriculum Learning and Fine-grained Fusion for Multimodal Rumor Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing multimodal rumor detection models overlook sample difficulty and order when training . Existing models overlook text-level difficulty, image-level and multimodal difficulty when training samples . |
| Approach: | They propose a curriculum learning framework that uses fine-grained fusion to detect rumors . they propose fusion-based methods that combine text and images to enhance semantic cohesion . |
| Outcome: | The proposed framework outperforms state-of-the-art models on English and Chinese benchmark datasets. |